Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tiling and TilesPatcher #89

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Tiling and TilesPatcher #89

wants to merge 2 commits into from
+542 −0

Conversation

spbui00
Copy link

@spbui00 spbui00 commented Sep 30, 2024

High level structure

Tiling node's key responsibilities

  • Divide the input frame into tiles based on the grid size and overlap.
  • Resize each tile to match the neural network’s input size.

Pathcer node's key responsibilities

  • bounding box extraction (decoding of neural network output)
  • scales and map the output bounding boxes to global image coordinates based on tile_index
  • performs Non-Maximum Suppression (NMS) (read more in paper or this Medium post) to eliminate duplicate detections and outputs the final set of bounding boxes
           ┌───────────────┐
           │               │
           │  ReplayVideo  │  # Inputs video frames
           │   or Camera   │
           │               │
           └──────┬────────┘
                  │
                  │ passthroughVideo 
                  │ (full frame)
                  ▼
           ┌───────────────┐
           │               │
           │    Tiling     │  # Splits frame into tiles
           │               │
           └──────┬────────┘
                  │
                  │ outputTiles
                  │ (n individual tiles)
                  ▼
           ┌───────────────┐
           │               │
           │ NeuralNetwork │  # Processes each tile, perform detection
           │               │
           └──────┬────────┘
                  │
                  │ outputDetections 
                  │ (n detections)
                  ▼
           ┌───────────────┐
           │               │
           │    Patcher    │  # Decode nn output and merges tile results
           │               │
           └──────┬────────┘
                  │
                  │ outputBoxes 
                  │ (merged boxes from n tiles)
                  ▼
           ┌───────────────┐
           │               │
           │    Display    │  # Shows final frame with bboxes from detections
           │               │
           └───────────────┘

@spbui00
initial commit with two nodes - Tiling and TilesPatcher | TilesPatche…
a26279a 
…r implemented to work with dai.ImgDetections
@github-actions github-actions bot added the release New version release label Sep 30, 2024
@spbui00 spbui00 changed the title initial commit with two nodes - Tiling and TilesPatcher Tiling and TilesPatcher Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tersekmatija tersekmatija Awaiting requested review from tersekmatija tersekmatija is a code owner

@jkbmrz jkbmrz Awaiting requested review from jkbmrz jkbmrz is a code owner

@kkeroo kkeroo Awaiting requested review from kkeroo kkeroo is a code owner

@klemen1999 klemen1999 Awaiting requested review from klemen1999 klemen1999 is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@spbui00 spbui00

Labels
release New version release
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@spbui00